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•  Blip-scan  radar  output  returns  are:  detect/no-detect,  or  {0, 1} 

•  Probability  of  detection  it  increases  as  range-to-target  decreases 

•  A  common  metric  is  R50  -  the  range  at  which  tt  =  50% 

•  A  common  question  is:  given  two  flights,  what  is  a  confidence 
interval  (C.l.)  for  the  difference  of  the  two  R50’s  ? 


•  Such  R(tt)  differences  are  non-linear  functions  of  the  parameters  of 
the  estimation  procedure;  its  own  distribution  is  hard  to  derive 

•  A  solution  to  find  a  C.l.  for  a  difference  is  to  use  a  Bootstrap 
procedure  =  a  non-parametric  simulation  approach 


•  Bootstrapping  works,  but  it  has  to  be  custom-generated  for  each 
different  problem  at  hand.  It’s  sometimes  preferable  to  have  a 
parametric  method.  We  develop  such  a  method  here  based  on  the 
Max.  Likelihood  Covariance  (inverse  of  the  Fisher  Information). 


3 


y  Logistic  curve  fit  to  Binary  data 


One  has  the  relation: 

-  Output  =  function(Range) 

But  output  is  binary  {0, 1}, 
and  we’d  rather  wish  to  find 
something  like: 

-  tt  =  function(Range) 

Transform  the  problem: 

y(R)  =  log  [tt/(1  -  tt)  ]  =  a  +  0R 

Now  we  have  a  linear  relation 
of  a  kind,  with 

n  =  exp(y)  /  [1  +  exp(y)] 


A  Logistic  curve 

Probability  is  on 
the  vertical  axis 


S/  Comparing  two  logistic  curves  Ip 

^ mm 


•  log  [tt/(1  -  tt)  ]  ==  logit(Tr)  is 

called  the  ‘logit’.  Log  ==ln 

•  The  graph  shows  two  such 
curves  (two  flights) 

•  At  n  =  0.5  the  blue  curve 
shows  R50  at  about  10  nm, 
the  red  curve  at  about  15  nm 

•  The  difference  is  about  5:  we 
want  a  95%  confidence 
interval  around  this  difference 


O 


o  i  | 

O  :  T 

S  i  i  r  r 

0  5  10  15  20 

Range  (nm) 

Two  logistic  curves 


Here,  P(detect)  decreases 
with  increasing  R 
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Estimation 


'fv, 
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•  Let  one  curve  be  estimated  with  non-linear  regression 
techniques  {generalized  linear  modeling)  to  give  the  equation 


logit(n )  =  a0  +  aiR 


and  let  the  other  curve  be  estimated  as 


logitfn)  =  /?0  + 


•  At  tt  =  0.5,  logit(iT)  =  log(0.5/0.5)  =  log(1)  =  0. 

So  for  the  first  curve  R0  =  -  ao/«i>  and  R1=  —  P0/Px  for  the  second. 
Their  estimated  difference  is  therefore  R0  -  R1  =  -( a0/ai  +  Po/Pi) 

•  Generalized  linear  modeling  uses  maximum  likelihood  estimation 
(MLE)  techniques  to  estimate  the  coefficients  of  the  models,  and  also 
gives  us  the  Covariance  Matrix  of  the  a  and  (3  parameters 

•  Call  this  covariance  matrix  /.  It  is  a  4x4  symmetric  matrix. 
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Confidence  Interval 


•  It  can  be  shown  (by  MLE  large-sample  theory)  that 

(R1  -  RO)  ~  Normal(  R1  -  RO,  hVh’ ) 
Where  V  is  the  covariance  matrix,  and  where 


) 


This  gives  us  the  (95%)  confidence  interval  that  we  desire  as: 


(R1  -  RO)  -1.96  x  hVh'  <  R1  -  RO  <  (R1  -  RO)  +  1.96  x  hVh 


C.l.  for  the  General  Case  Ip 

^ -  mm 

•  So  far,  we’ve  developed  a  Cl  for  R50;  that  is,  where  tt  =  0.5 

•  We  can  get  a  Cl  for  any  value  of  tt  in  (0, 1)  by  replacing  the  ‘h’ 
we  used  in  the  above  slide  with 

h  =  /—  ~  ZEo  \ 

V  «i2  )l 

where  yc  is  the  estimate  of  the  logit(Tr)  at  the  new  value  of  tt. 

•  The  above  theory  depends  on  the  assumption  that  the  two 
flights  gave  independently- estimated  curves,  and  the  curves 
do  not  cross  over  each  other. 
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Bootstrap  Test 
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We  ran  bootstrap  simulations  against  our  analytic  technique  for 
tt  =  p  =  0.2, 0.5,  and  0.8  to  see  how  we  compared,  and  also 
compared  our  F.l.  method  against  Schwenke  &  Milliken’s 


At  probability 

Method  1 

Method  2 

Method  3 

Method  4 

Analytical  95%  C.l.  Bootstrap  95%  C. I .  Bias  Corrected  Normal  approx, 
based  on  Fisher  Bootstrap  95%  C.l.  method  of 

Information  Schwenke  &  Milliken 


[2.7,  3.7] 
[4.8,  5.9] 
[6.9,  8.0] 


[2.5,  3.8] 
[4.8,  5.9] 
[6.7,  8.3] 


[2.56,  3.9] 
[4.9,  5.9] 
[6.7,  8.3] 


[2.4,  4.0] 
[4.5,  6.2] 
[6.7,  8.3] 


Our  method  produces  intervals  very  close  to  the  Bootstrap  © 

Note:  These  are  all  large-sample  results 
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Small-sample  Test 
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We  looked  at  a  smaller  sample  size  (n=200),  and  compared  our 
‘Analytical’  method  against  Schwenke  &  Milliken’s.  (The  data  is 
randomly  sampled  from  the  original  bootstrap  data  set). 


p  =0.5 
n=200 

Method  1 

Method  2 

Method  3 

Run 

Analytical  95% 
C.l. 

based  on  Fisher 
Information 

Schwenke  & 
Milliken’s  method 
95%  C.l. 

Bias  Corrected 
Bootstrap  95%  C.l. 

#1 

[0.4,  5.5] 

[-1.8,  7.8] 

[4.9,  5.9  ] 

#2 

[3.8,  8.1] 

[1.2, 10.8] 

[4.9,  5.9  ] 

#3 

[2.1, 6.0] 

[0.5,  7.6] 

[4.9,  5.9  ] 

Our  method  produces  narrower  intervals  than  S&M 

NOTE:  This  conclusion  is  based  on  just  3  runs;  more  extensive  tests  are  planned 
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Summary-  R50  &  the  Logistic  Model 
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•  We  looked  at  a  Cl  on  the  difference  between  R50  points  for 
two  independent  flights 

•  We  extended  the  results  to  the  difference  between  two  Rp 
points,  where  0  <  p  <  100 

•  Our  ‘analytic’  method  is  based  on  the  covariance  matrix 
generated  from  the  MLE  procedure  of  generalized  regression 

•  We  compared  Cl’s  of  our  method  to  the  ‘true’  Cl  generated  by 
a  large  bootstrap  sample  (n  =  4000  scans/flight),  and  also  to 
an  alternate  method  by  Schwenke  &  Milliken  (1991) 

•  We  further  looked  at  the  comparative  results  for  a  ‘small’ 
sample  (n  =  200  scans/flight). 
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